Lower Bounds for Learning Discrete Distributions

We study the model of learning discrete probability distributions which was introduced by Kearns, Mansour, Ron, Rubinfeld, Schapire, and Sellie in 13]. Our main results are a lower bound on computational sample complexity and a lower bound on hypothesis size for this learning model. We deene a class of eeciently learnable distributions which has the following interesting property: while a computationally unbounded learning algorithm can learn the class from O(1) examples, the computational sample complexity of the class is essentially (1==) in the sense that any polynomial-time learning algorithm must use at least this many examples. The construction is based on the existence of any length-preserving one-way permutation. We also prove a lower bound on hypothesis size. Based on the existence of any one-way function, we construct a class of probability distributions learnable in polynomial time from ~ O(1== 2) examples and prove that any polynomial-time learning algorithm for this class must use hypotheses of size essentially (1== 1=2): This answers a conjecture posed by Kearns et al. in 13]. Note: After completing a draft of this paper, it was brought to our attention that Moni Naor 16] had already established, via a slightly diierent construction, results which are essentially equivalent to our main result of Section 5 (the lower bound on hypothesis size). We thank the unknown referee who brought 16] to our attention.